1. Field of the Invention
The present invention relates to video signal processing. More specifically, the present invention relates to an outline emphasis process for a video signal, which is used in digital camcorders and the like.
2. Description of the Related Art
Generic imaging apparatuses such as digital camcorders include an imaging means, a video signal processing means, and an encoding means.
Through imaging optics, the imaging means allows subject light (light from a subject) to be converged on a solid-state imaging device for a photoelectric conversion, and generates a video signal. To the video signal which has been obtained by the imaging means, the video signal processing means applies various kinds of signal processing for improving the image quality. Then, by an encoding method such as MPEG2, the encoding means applies a data amount compression to the video signal which has been subjected to the signal processing.
Since the conditions for shooting a subject may be various, the video signal processing means of an imaging apparatus performs various kinds of control for stably obtaining a good image quality under any shooting condition. The examples are aperture, shutter speed, focus, handshake correction, and zoom operation, which are controlled for the imaging means. An automatic gain control (AGC) is also performed. Furthermore, for an improved image quality, the video signal processing means also performs a gamma correction process, an outline emphasis process, and the like.
For example, Japanese Laid-Open Patent Publication No. 2-63271 and Japanese Laid-Open Patent Publication No. 2002-64745 disclose a device which may typically be a camera.
FIG. 10 is a construction diagram showing the construction of a conventional imaging apparatus.
The imaging apparatus includes an imaging section 20, various control sections 21 to 25, an imaging control section 42, and a video signal processing section 44.
The subject light which has entered imaging optics 19 is converged onto the imaging device through lenses 10, 11, 13, and 14, and an iris 12 of the imaging optics 19. Then, the subject light is converted into an analog video signal by the imaging device 15, a CDS circuit 16 and an AGC circuit 17, and is further converted into a digital video signal by an A/D converter 18.
Next, the converted digital video signal is input to a video signal processing section 44. A gamma correction circuit 40 of the video signal processing section 44 applies a gamma correction process to the input digital video signal. Next, an outline emphasis processing section 41 applies an outline emphasis process to the video signal that has experienced gamma correction. Specifically, the outline emphasis processing section 41 multiplies the video signal which has been subjected to the gamma correction process by an externally-received gain value, thus amplifying the outline component signal to an appropriate level. The outline emphasis processing section 41 outputs the resultant outline component signal to the subsequent stages via an output terminal 43.
For the aforementioned processes by the imaging optics 19, the imaging control section 42 controls the various control sections 21 to 25. Specifically, the imaging control section 42 controls a zoom control section 21, an iris control section 22, an OIS (Optical Image Stabilizer) control section 23, a shutter control section 24, and an AGC control section 25. Through such control, a stable image quality is automatically obtained under any shooting condition.
The conventional outline emphasis processing method has problems in that: it is impossible to perform an effective outline emphasis process that supports various shooting conditions; a balance cannot be established between deterioration in image quality due to encoding and improvement in image quality due to outline emphasis; and use of a plurality of pieces of control information adds to the circuit scale, e.g., memory, thus increasing the cost. Specifically, they are as follows.
As an first example, the aforementioned problems become significant when the illuminance of the subject is lowered.
Firstly, when the illuminance of the subject that is input to the imaging device 15 is bright, a video image having a high contrast in the outline portion is obtained, so that the effect of outline emphasis will be eminent. Moreover, since there are many high-luminance regions within the screen, a good signal to noise (S/N) ratio is also obtained. Thus, even if a large outline emphasis gain value is set, there is no unwanted amplification of noise, and it is possible to achieve an improved image quality.
On the other hand, the S/N ratio will gradually be deteriorated as the illuminance decreases. In particular, noise will become conspicuous in flat portions that are dark. However, during the outline emphasis process, a relatively large gain value is universally applied irrespective of the illuminance, by which the outline component signal is amplified. This will also allow the noise component to be amplified, thus resulting in a very uncomely image quality. Moreover, when the illuminance is low, contrast will lower in the outline portion; therefore, even if an outline emphasis process is performed, its effect will not be eminent.
As a second example, when a motion occurs in the subject due to a motion of the imaging optics or the imaging apparatus main body such as zoom or panning, the aforementioned problems will also be significant.
Firstly, in a still shooting where the imaging optics and the imaging apparatus main body are not moving, it is easy to focus on the subject and the outline portion will be clearly imaged. If an outline emphasis process is then performed, the outline will become emphasized furthermore clearly, whereby an improved image quality can be achieved.
However, when shooting is conducted while panning the camera or zooming, the subject will be moving so that it is difficult to attain focus, and the outline portion will be blurred. In other words, there is less high frequency component, so that the effect of outline emphasis will not be eminent even if an outline emphasis process is performed.
Furthermore, there is also a fear of emphasizing unwanted signal components. For example, in the case where a three-dimensional noise reduction circuit for noise reduction (not shown) is provided, unwanted components that do not belong to the subject may appear near an outline portion that has moved, e.g., afterimage or pseudo outline. The outline emphasis process will also emphasize such unwanted component at the same time. Furthermore, in a low-illuminance situation where the S/N ratio is bad, only the noise component will be emphasized, thus resulting in a deterioration in the image quality.
In Japanese Laid-Open Patent Publication No. 2-63271 above, an outline emphasis method for reducing deterioration in the image quality under a low illuminance is proposed. This is a technique of controlling an outline emphasis factor in synchronization with an AGC gain value that is applied to the AGC circuit 17.
Although this can address problems under a very low illuminance, such as when an intensive outline emphasis process will be performed by the AGC circuit 17, it still leaves some problems in the case of a illuminance which is not so low as the illuminance at which the AGC circuit 17 will begin operating but which is objectively quite low.
Even before AGC control is begun, an illuminance adjustment in the imaging section is being performed by controlling the aperture level, shutter speed, etc., of the iris. For example, when the iris is open with a shutter speed of 1/60 seconds, the illuminance is so low that there will be significant lowering of resolution and deterioration in the S/N ratio. Thus, an improvement with respect to a single condition will leave problems as to other conditions, and it is necessary to totally consider the conditions for a plurality of pieces of camera control information.
As an example of using a plurality of pieces of camera control information, there exists a technique which is disclosed in Japanese Laid-Open Patent Publication No. 2002-64745, supra. Under the method shown in this document, setting tables are retained in memory such that parameter sets will be loaded according to case handling of each parameter based on a plurality of conditions. However, with this construction, the amount of tables will increase with an increase in the number of pieces of imaging control information that are used, thus inducing a considerable increase in cost, e.g., increased memory amount.
Moreover, performing an outline emphasis process will lead to yet other problems.
For example, when moving pictures are shot with a camcorder, a video signal will be encoded, and written to a storage medium as moving picture data. If an outline emphasis process is performed before an encoding process, the block noise may increase during an encoding process which is based on this process result, thus resulting in a deterioration in the image quality. It may be said that the outline emphasis process which is meant to provide a better image quality (sharpness) is serving as a cause for a subsequent deterioration in image quality, thus producing a converse effect.
Hereinafter, with reference to FIG. 11, the reasons for the increased block noise will be described. FIG. 11 shows the construction of an encoding section 81 of a conventional imaging apparatus. As an example, a construction which performs encoding by the MPEG2 method is illustrated.
Hereinafter, a flow of processing that is performed in the encoding section 81 will be described.
First, a video signal is input at an input terminal 80. It is assumed that the input video signal is a video signal that has experienced correction.
Based on a predetermined gain value, the outline emphasis processing section 41 applies an outline emphasis process to the video signal. The video signal which has been subjected to the outline emphasis is retained in a sorting memory 51.
The sorting memory 51 outputs encoded images which are sorted in the order of encoding. From an encoded image which is output from the sorting memory 51 and a reference image which is output from a reference image memory 52, a motion-compensated prediction section 53 performs a motion compensation, and generates a predictable image. A subtracter 54 determines an image difference between this predictable image and the encoded image, and outputs it to a DCT circuit 55. The image difference is subjected to an orthogonal transform by the DCT circuit 55, and is quantized by a quantization section 56 according to a quantization parameter which is supplied from a rate control section 61. The result thereof is supplied to a variable-length coding section 60, and also to an inverse quantization section 57. The data which is fed to the variable-length coding section 60 is variable-length encoded, and recorded onto a storage medium 82. On the other hand, the data which is fed to the inverse quantization section 57 is inverse-quantized according to the aforementioned quantization parameter, subjected to an inverse orthogonal transform by an inverse DCT circuit 58, added up with the aforementioned predictable image, and is stored to the reference image memory. Note that only images which are selected for the reference image are stored to the reference image memory.
The rate control section 61 controls the level of quantization so that the encoded data will fit within the target recording bitrate that has been set. The control of quantization level is performed by changing the size of the quantization parameter.
The information amount of an input image is always fluctuating incessantly. Generally speaking, an image having many flat portions and little motion has a small information amount, whereas a complicated image having many outline portions or an image having a lot of motion has a large information amount. In order to fit an image having a large information amount within a predetermined recording bitrate, quantization must be performed with a greater quantization parameter than in the case of fitting an image having a small information amount within the same recording bitrate. As a result, block noises due to increased quantization errors will become significant.
The outline emphasis process by the outline emphasis processing section 41 increases the signal level of the high frequency component. The end result of this is that the high frequency component to be eliminated through the encoding process is less likely to be eliminated, and will remain in abundance. Then, the encoding section will act to further increase the quantization parameter so as to be commensurate with the target recording bitrate.
Thus, even though an effect of improving the sharpness through an outline emphasis process is being expected, a converse effect will occur in that the block noise is increased and the image quality is deteriorated. This phenomenon becomes particularly significant when the recording bitrate is too low relative to the information amount of the image.
Moreover, the rate control section 61 is measuring the resultant code amount for each encoded image, in order to perform a rate control. Then, in the case of performing a variable rate control (VBR), when the image information amount is increased, not a control of merely increasing the quantization parameter is performed, but a control of slightly increasing the quantization parameter while also permitting the resultant code amount to somewhat exceed the recording bitrate is performed. Therefore, the level of change in the image quality due to encoding (the influence of deterioration in image quality) will be manifested in two parameters, i.e., the quantization parameter and the amount of increase in the resultant code.
Moreover, in the case where a quantization control is performed in accordance with the magnitude of a cumulative error amount, which is a difference between the recording capacity that is consumed at the target recording bitrate and the recording capacity that is consumed at the actually-occurring code amount, the resultant code amount will be limited based on the cumulative error amount. Therefore, the level of deterioration in image quality will also be manifested in another parameter, i.e., the cumulative error amount.
Also, the image quality is naturally a function of how high the target recording bitrate is. The reason is that the aforementioned increase in block noise and consequent deterioration in image quality will be significant when the recording bitrate is too low relative to the information amount of the image. Therefore, the level of deterioration in image quality is also manifested in another parameter, i.e., the target recording bitrate.
As described above, the image quality cannot be improved through an outline emphasis process alone, and the influences on various parameters must also be taken into consideration.